Large language models encode clinical knowledge.

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

On the accuracy of T1 mapping: searching for common ground.

PURPOSE: There are many T1 mapping methods available, each of them validated in phantoms and reporting excellent agreement with literature. However, values in literature vary greatly, with T1 in white matter ranging from 690 to 1100 ms at 3 Tesla. This brings into question the accuracy of one of the most fundamental measurements in quantitative MRI. Our goal was to explain these variations and look into ways of mitigating them. THEORY AND METHODS: We evaluated the three most common T1 mapping methods (inversion recovery, Look-Locker, and variable flip angle) through Bloch simulations, a white matter phantom and the brains of 10 healthy subjects (single-slice). We pooled the T1 histograms of the subjects to determine whether there is a sequence-dependent bias and whether it is reproducible across subjects. RESULTS: We found good agreement between the three methods in phantoms, but poor agreement in vivo, with the white matter T1 histogram peak in healthy subjects varying by more than 30% depending on the method used. We also found that the pooled brain histograms displayed three distinct white matter peaks, with Look-Locker consistently underestimating, and variable flip angle overestimating the inversion recovery T1 values. The Bloch simulations indicated that incomplete spoiling and inaccurate B1 mapping could account for the observed differences. CONCLUSION: We conclude that the three most common T1 mapping protocols produce stable T1 values in phantoms, but not in vivo. To improve the accuracy of T1 mapping, we recommend that sites perform in vivo validation of their T1 mapping method against the inversion recovery reference method, as the first step toward developing a robust calibration scheme.

Parameter estimation approach to banding artifact reduction in balanced steady-state free precession.

PURPOSE: The balanced steady-state free precession (bSSFP) pulse sequence has shown to be of great interest due to its high signal-to-noise ratio efficiency. However, bSSFP images often suffer from banding artifacts due to off-resonance effects, which we aim to minimize in this article. METHODS: We present a general and fast two-step algorithm for 1) estimating the unknowns in the bSSFP signal model from multiple phase-cycled acquisitions, and 2) reconstructing band-free images. The first step, linearization for off-resonance estimation (LORE), solves the nonlinear problem approximately by a robust linear approach. The second step applies a Gauss-Newton algorithm, initialized by LORE, to minimize the nonlinear least squares criterion. We name the full algorithm LORE-GN. RESULTS: We derive the Cramér-Rao bound, a theoretical lower bound of the variance for any unbiased estimator, and show that LORE-GN is statistically efficient. Furthermore, we show that simultaneous estimation of T1 and T2 from phase-cycled bSSFP is difficult, since the Cramér-Rao bound is high at common signal-to-noise ratio. Using simulated, phantom, and in vivo data, we illustrate the band-reduction capabilities of LORE-GN compared to other techniques, such as sum-of-squares. CONCLUSION: Using LORE-GN we can successfully minimize banding artifacts in bSSFP.

Effect of intratonsillar injection of steroids on the palatine tonsils of rabbits.

OBJECTIVES/HYPOTHESIS: Nasal steroids may significantly improve nasal obstructive symptoms with a reduction of adenoid size in children, but they do not consistently yield the same concurrent effect on enlarged palatine tonsils. Failure of nasal steroids to decrease the size of palatine tonsils is believed to be attributable to location and washout by saliva. The purpose of this study was to determine if direct application of steroid via intratonsillar injection would reduce the size of palatine tonsils in the rabbit model. STUDY DESIGN: Prospective animal study. METHODS: Eight rabbits (16 tonsils) were administered intratonsillar injections of fluticasone (n = 8, 1 mg/ml) or saline (n = 8, 0.1 ml) on days 0, 3, 7, 10, 14, and 17. Two rabbits (4 tonsils) received a single steroid injection to compare single versus multiple steroid injections. The rabbit's tonsil size was measured before each injection. After the fifty injections, the tonsils were harvested for histologic analysis. RESULTS: A total of 16 tonsils were analyzed. After five steroid injections, the reduction (-7.7 mm(2) ± 4.27) in size was statistically significant when compared to reduction (6.12 mm(2) ± 6.57) in the saline injected group (P = 0.001). Repeated steroid injection was more potent than a single injection (-3.00 mm(2) ± 3.08) in reducing the size (P = 0.006). In histologic analysis, tonsils after repeated steroid injections were significantly smaller than saline-injected tonsils (P = 0.014), without obvious lymphoid follicles. CONCLUSION: Repeated focal tonsillar injections of corticosteroids significantly reduced the size of palatine tonsils as compared to saline-injected controls. A single injection of corticosteroids appears to be effective, but not as effective, as multiple injections. LEVEL OF EVIDENCE: N/A.

Cost-effectiveness landscape analysis of treatments addressing xerostomia in patients receiving head and neck radiation therapy.

Head and neck (H&N) radiation therapy (RT) can induce irreversible damage to the salivary glands thereby causing long-term xerostomia or dry mouth in 68%-85% of the patients. Not only does xerostomia significantly impair patients' quality-of-life (QOL) but it also has important medical sequelae, incurring high medical and dental costs. In this article, we review various measures to assess xerostomia and evaluate current and emerging solutions to address this condition in H&N cancer patients. These solutions typically seek to accomplish 1 of the 4 objectives: (1) to protect the salivary glands during RT, (2) to stimulate the remaining gland function, (3) to treat the symptoms of xerostomia, or (4) to regenerate the salivary glands. For each treatment, we assess its mechanisms of action, efficacy, safety, clinical utilization, and cost. We conclude that intensity-modulated radiation therapy is both the most widely used prevention approach and the most cost-effective existing solution and we highlight novel and promising techniques on the cost-effectiveness landscape.

Theranostic effect of serial manganese-enhanced magnetic resonance imaging of human embryonic stem cell derived teratoma.

Although human embryonic stem cell (hESC) hold therapeutic potential, teratoma formation has deterred clinical translation. Manganese (Mn(2+)) enters metabolically active cells through voltage-gated calcium channels and subsequently, induces T(1) shortening. We hypothesized that serial manganese-enhanced MRI would have theranostic effect to assess hESC survival, teratoma formation, and hESC-derived teratoma reduction through intracellular accumulation of Mn(2+). Firefly luciferase transduced hESCs (hESC-Lucs) were transplanted into severe combined immunodeficient mouse hindlimbs to form teratoma. The chemotherapy group was injected with MnCl(2) intraperitoneally three times a week. The control group was given MnCl(2) only prior to manganese-enhanced MRI. Longitudinal evaluation by manganese-enhanced MRI and bioluminescence imaging was performed. The chemotherapy group showed significant reduction in the teratoma volume and luciferase activity at weeks 6 and 8. Histology revealed increased proportion of dead cells and caspase 3 positive cells in the chemotherapy group. Systemic administration of MnCl(2) enabled simultaneous monitoring and elimination of hESC-derived teratoma cells by higher intracellular accumulation of Mn(2+).

Real-time motion correction for high-resolution larynx imaging.

Motion--both rigid-body and nonrigid--is the main limitation to in vivo, high-resolution larynx imaging. In this work, a new real-time motion compensation algorithm is introduced. Navigator data are processed in real time to compute the displacement information, and projections are corrected using phase modulation in k-space. Upon automatic feedback, the system immediately reacquires the data most heavily corrupted by nonrigid motion, i.e., the data whose corresponding projections could not be properly corrected. This algorithm overcomes the shortcomings of the so-called diminishing variance algorithm by combining it with navigator-based rigid-body motion correction. Because rigid-body motion correction is performed first, continual bulk motion no longer impedes nor prevents the convergence of the algorithm. Phantom experiments show that the algorithm properly corrects for translations and reacquires data corrupted by nonrigid motion. Larynx imaging was performed on healthy volunteers, and substantial reduction of motion artifacts caused by bulk shift, swallowing, and coughing was achieved.

In vivo molecular MRI of cell survival and teratoma formation following embryonic stem cell transplantation into the injured murine myocardium.

Embryonic stem cells (ESCs) have shown the potential to restore cardiac function after myocardial injury. Superparamagnetic iron oxide nanoparticles (SPIO) have been widely employed to label ESCs for cellular MRI. However, nonspecific intracellular accumulation of SPIO limits long-term in vivo assessment of the transplanted cells. To overcome this limitation, a novel reporter gene (RG) has been developed to express antigens on the ESC surface. By employing SPIO-conjugated monoclonal antibody against these antigens (SPIO-MAb), the viability of transplanted ESCs can be detected in vivo. This study aims to develop a new molecular MRI method to assess in vivo ESC viability, proliferation, and teratoma formation. The RG is designed to express 2 antigens (hemagglutinin A and myc) and luciferase on the ESC surface. The two antigens serve as the molecular targets for SPIO-MAb. The human and mouse ESCs were transduced with the RG (ESC-RGs) and transplanted into the peri-infarct area using the murine myocardial injury model. In vivo MRI was performed following serial intravenous administration of SPIO-MAb. Significant hypointense signal was generated from the viable and proliferating ESCs and subsequent teratoma. This novel molecular MRI technique enabled in vivo detection of early ESC-derived teratoma formation in the injured murine myocardium.

A molecular MRI probe to detect treatment of cardiac apoptosis in vivo.

Cell death by apoptosis is critical in myocardial diseases, and noninvasive detection of early, reversible apoptosis might be useful clinically. Exogenous Annexin-V (ANX) protein binds membrane phosphatidylserine, which is externalized in early apoptosis. A molecular MRI probe was constructed with superparamagnetic iron oxide (SPIO) conjugated to recombinant human ANX (ANX-SPIO). Apoptosis was induced with doxorubicin, a cardiotoxic cancer drug, in culture in neonatal rat ventricular myocytes, cardiac fibroblasts, and mesenchymal stem cells, and in vivo in the mouse heart. ANX-SPIO was validated using T2*-weighted 3T MRI. ANX-SPIO produced T2* signal loss, reflecting iron content, that correlated highly with independent apoptosis markers; bound with high affinity to apoptotic myocytes by competition assay (Ki 69 nM); detected apoptosis in culture much earlier than did TUNEL stain; and revealed fibroblast resistance to apoptosis. With apoptosis in vivo, ANX-SPIO produced diffuse myocardial T2* signal loss that correlated with increased iron stain and caspase activity. Treatment with an alpha-1-adrenergic agonist in vivo reversed apoptosis and eliminated the ANX-SPIO MRI signal. It is concluded that cardiac MRI of ANX-SPIO detects early, nonischemic cardiac apoptosis in culture and in vivo, and can identify reversibly injured cardiac cells in diseased hearts, when treatment is still possible.

FeCo/graphite nanocrystals for multi-modality imaging of experimental vascular inflammation.

BACKGROUND: FeCo/graphitic-carbon nanocrystals (FeCo/GC) are biocompatible, high-relaxivity, multi-functional nanoparticles. Macrophages represent important cellular imaging targets for assessing vascular inflammation. We evaluated FeCo/GC for vascular macrophage uptake and imaging in vivo using fluorescence and MRI. METHODS AND RESULTS: Hyperlipidemic and diabetic mice underwent carotid ligation to produce a macrophage-rich vascular lesion. In situ and ex vivo fluorescence imaging were performed at 48 hours after intravenous injection of FeCo/GC conjugated to Cy5.5 (n = 8, 8 nmol of Cy5.5/mouse). Significant fluorescence signal from FeCo/GC-Cy5.5 was present in the ligated left carotid arteries, but not in the control (non-ligated) right carotid arteries or sham-operated carotid arteries (p = 0.03 for ligated vs. non-ligated). Serial in vivo 3T MRI was performed at 48 and 72 hours after intravenous FeCo/GC (n = 6, 270 µg Fe/mouse). Significant T2* signal loss from FeCo/GC was seen in ligated left carotid arteries, not in non-ligated controls (p = 0.03). Immunofluorescence staining showed colocalization of FeCo/GC and macrophages in ligated carotid arteries. CONCLUSIONS: FeCo/GC accumulates in vascular macrophages in vivo, allowing fluorescence and MR imaging. This multi-functional high-relaxivity nanoparticle platform provides a promising approach for cellular imaging of vascular inflammation.

A robust methodology for in vivo T1 mapping.

In this article, a robust methodology for in vivo T(1) mapping is presented. The approach combines a gold standard scanning procedure with a novel fitting procedure. Fitting complex data to a five-parameter model ensures accuracy and precision of the T(1) estimation. A reduced-dimension nonlinear least squares method is proposed. This method turns the complicated multi-parameter minimization into a straightforward one-dimensional search. As the range of possible T(1) values is known, a global grid search can be used, ensuring that a global optimal solution is found. When only magnitude data are available, the algorithm is adapted to concurrently restore polarity. The performance of the new algorithm is demonstrated in simulations and phantom experiments. The new algorithm is as accurate and precise as the conventionally used Levenberg-Marquardt algorithm but much faster. This gain in speed makes the use of the five-parameter model viable. In addition, the new algorithm does not require initialization of the search parameters. Finally, the methodology is applied in vivo to conventional brain imaging and to skin imaging. T(1) values are estimated for white matter and gray matter at 1.5 T and for dermis, hypodermis, and muscle at 1.5 T, 3 T, and 7 T.

In vivo high-resolution magnetic resonance skin imaging at 1.5 T and 3 T.

As a noninvasive modality, MR is attractive for in vivo skin imaging. Its unique soft tissue contrast makes it an ideal imaging modality to study the skin water content and to resolve the different skin layers. In this work, the challenges of in vivo high-resolution skin imaging are addressed. Three 3D Cartesian sequences are customized to achieve high-resolution imaging and their respective performance is evaluated. The balanced steady-state free precession (bSSFP) and gradient echo (GRE) sequences are fast but can be sensitive to off-resonance artifacts. The fast large-angle spin echo (FLASE) sequence provides a sharp depiction of the hypodermis structures but results in more specific absorption rate (SAR). The effect of increasing the field strength is assessed. As compared to 1.5 T, signal-to-noise ratio at 3 T slightly increases in the hypodermis and almost doubles in the dermis. The need for fat/water separation is acknowledged and a solution using an interleaved three-point Dixon method and an iterative reconstruction is shown to be effective. The effects of motion are analyzed and two techniques to prevent motion and correct for it are evaluated. Images with 117 x 117 x 500 microm(3) resolution are obtained in imaging times under 6 min.